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Guidance method and device 
Field of the invention 

The present invention generally relates to the display of electronic 
5 equipment such as mobile phones and more particularly to the browsing and 
reading of a large document on a display screen. 

Background of the invention 

Today portable electronic equipment, such as mobile phones and 
10 personal digital assistance devices PDA, are getting ever-smaller. However, 
when the size of the portable equipment is reduced also the size of the 
equipment's display is reduced. On the other hand, there is an increasing 
need to provide services with text and pictures to subscribers. Thus the 
screen of the display is usually smaller in size than the actual document, i.e. 
15 only a portion of the document is visible at a time. It is clear that reading a 
large document from a small display screen is rather difficult. 

Different kinds of mechanisms have been developed to ease 
reading from a small display screen. Some of them are considered briefly in 
the following. 

20 In some small-sized, hand-held devices which can be used while 

being carried, scrolling the view on the display screen is based on the loca- 
tion (at least one of x, y, or z coordinates) and/or orientation (at least one of 
azimuth, elevation, roll) of the device. In such a motion-control environment, 
the portion displayed on the display screen of the hand-held device depends 

25 constantly on the movements of the hand. Due to the fact that the hand is 
inaccurate, some unwanted effects arise, such as jitter or bouncing, which 
makes reading even more difficult. The same kind of unwanted effects may 
arise also when the hand-held device is used in a moving vehicle. 

In order to avoid the above drawbacks, a method has been de- 

30 vised whereby the direction of the scrolling the data can be locked. However, 
one drawback with this method is that it allows browsing of the data in one 
direction only, i.e. it does not take into account the direction of the text lines 
in the data. 

Methods with scroll bars permitting movements of the text either 
35 horizontally or vertically do not have jitter. That is, scroll bars can be used to 
permit the user to change the portion of the view on the display screen hori- 
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zontally by sliding the horizontal scroll bar and vertically by sliding the verti- 
cal scroll bar. However, a drawback with such methods is that scrolling is 
always either horizontal or vertical, i.e. no other direction is possible. 

At the moment there is no method available that reduces the er- 
5 rors arising from the bouncing movements around the text lines when read- 
ing a document on the display screen of a portable device assisted by a 
motion-control method. 

Summary of the invention 

1 0 An objective of the present invention is to provide a guidance method 

for scrolling the display on a screen of a hand-held device having motion- 
controlled scrolling and a hand-held device performing the method. The view 
shown on the display is a portion of larger content comprising text lines that 
are too long to be displayed entirely on the small display. 

15 The guidance method should be adapted to follow accurately the 

reading direction of the text lines from the beginning of the text lines to the 
end of the text lines. The reading direction should remain as uniform as 
possible during the entire reading event, i.e. unwanted effects such as jitter 
or bouncing of the text lines should not arise during the reading event. 

20 A typical situation may be when a user is browsing a large full-size 

image or a document by scrolling the view shown on a mobile phone's dis- 
play using motion-control scrolling. 

The guidance method discovers when the user stops browsing 
and starts to read text lines. Then the need for guidance is examined. If the 

25 result of examination indicates that the user is reading the text lines, the 
scrolling direction is guided to follow the reading direction of the text lines. 
The guidance is invisible to the user. 

The objective is achieved in the manner described in the inde- 
pendent claims. 

30 The guidance method comprises three main steps: 1) the direction 

of the text lines shown on the display screen is periodically examined; 2) the 
movements of the mobile phone are periodically examined and, based on 
the movements, it is concluded whether the user is reading the text lines; 3) 
if the result of examination in step 2 indicates that the user is reading the text 

35 lines, the scrolling direction is reinforced in the direction of the text line so 
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long as the difference between the current scrolling direction and the text line 
direction is less than a predetermined threshold value. 

Brief description of the drawings 

5 The invention is described more closely with reference to the ac- 

companying drawings, in which 

FIG. 1a-b illustrates a page of the document to be read on a display of a 
small portable device, 
10 FIG. 2 is a flowchart illustrating definitions of the direction of text lines on 
the display, 

FIG. 3 is a flowchart illustrating the method according to the invention, 
FIG. 4 illustrates the inventive idea with the help of a full-size image to be 
read on a display of a small portable device, 
15 FIG. 5a-b show an example of the determination of coordinates on the full- 
size image, and 
FIG. 6a-b demonstrate how filtering is carried out. 

Detailed description of the invention 

20 The invention can be applied to a terminal having a small display, 

at least one sensor such as a magnetic pulse sensor, a acceleration trans- 
ducer or gyroscope which is mounted in the back of the display, and a char- 
acter recognition method or, alternatively, some other method for recognizing 
the direction of the text lines. 

25 The principle of the invention and the embodiments are described 

using a mobile phone as an example, but the same principle can, of course, 
be applied to any kind of terminal. 

FIG. 1a shows a typical situation where a user is browsing a large 
full-size image 10 on the display of a mobile phone. In this context the full- 

30 size image refers to a whole document or page, of which only a part can be 
displayed on the actual screen of the terminal. The actual display of the 
mobile phone is illustrated by a dotted square 16 on the document page for 
visualizing the proportions of the display to the document. Thus the user is 
reading a document whose actual size is larger than the physical size of the 

35 display. The document page has been divided into several columns consist- 
ing of text sections 11-13 and non-text sections 14-15, such as pictures. 
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Lines in the text column 13 are in a different direction than in the other two 
text columns 11 and 12. 

FIG. 1b is an enlargement of the circled area in FIG. 1a. As seen, 
only a portion of the data is seen on the display at a time, i.e. a line of the 
5 document does not fit from beginning to end on the display. Scrolling direc- 
tions are shown by double-ended arrows. Other scrolling directions are also 
possible, e.g. in the diagonal direction. 

It is assumed that motion-controlled scrolling is used. In order to 
browse the document, the data is displayed proportionally to the orientation 

10 or location of the mobile phone. That is, data scrolling in the vertical direction 
when the mobile phone is tilted vertically, and correspondingly, the data 
scrolls horizontally when the mobile phone is tilted horizontally. 

When a piece of interesting text 17 appears on the display and 
the user wants to go into the matter in greater detail, i.e. to read through the 

15 whole document or a chapter concerning the matter, the portion of the view 
displayed is moved to the beginning of the document or appropriate chapter 
by tilting the mobile phone in a suitable way. For example, a movable cursor 
is used to indicate a position of interest on the display surface by tilting the 
mobile phone. Thus the cursor moves horizontally when the mobile phone is 

20 tilted towards the vertical axis of the display, and correspondingly, the cursor 
moves vertically when the mobile phone is tilted towards the horizontal axis 
of the display. 

Thus, the display shown on the screen depends on the motion of 
the hand. However, as stated above, hand movements are inaccurate. With- 

25 out accurate guidance, unwanted jitter arises, making the reading uncom- 
fortable and slow. This drawback is eliminated by the guidance method de- 
scribed in more detail in the following. 

The guidance method comprises three main steps: 1) the direction 
of the text lines shown on the display screen is periodically examined; 2) the 

30 movements of the mobile phone are periodically examined and, based on 
the movements, it is concluded whether the user is reading the text lines; 3) 
if the result of examination in step 2 indicates that the user is reading the text 
lines, the scrolling direction is reinforced in the direction of the text line so 
long as the difference between the current scrolling direction and the text line 

35 direction is less than a predetermined threshold value. 
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FIG. 2 shows as a flowchart an example of the implementation of 
examination of the direction of text lines. 

It is assumed here that a user scrolls documents on the mobile 
phone display screen in a way that is described in association with FIG. 1a 
5 and 1b. 

At stage 20, the first task is to analyze roughly whether there is 
text in the current portion on display. If no text is detected, free scrolling of 
the data is allowed at stage 21, and the control is returned to stage 20. 

There are numerous implementation alternatives for analysis. One 
10 alternative is that the portion of the data that is seen on the display screen is 
first analyzed by defining the major components, i.e. whether the type of 
component represents the text or non-text. Some character recognition 
method such as an optical character recognition method OCR, can be used. 
In some cases information about the type and size of the document, and the 
15 type and size of the font may already be included in the document, in which 
case no character recognition is needed. 

At the next stage 22, the direction of the text lines is defined. In 
FIG. 1a most of the page is covered with text. When this page is analyzed by 
the character recognition method the direction of text lines is defined to be 
20 horizontal. However, if the column 13 had been shown on the display, the 
text lines would have been defined to be vertical. 

One alternative for the direction definition of the text lines can be 
created on the basis of the middle point of the current portion of view dis- 
played. This method will be considered later in greater detail. 
25 Another alternative utilizes image analysis and character recogni- 

tion. The direction is defined on the basis of features of the analyzed data. 

The definition results concerning the direction of the text lines are 
saved at stage 23. The information also includes current time information. 
The above-described analysis and definition are came out periodically at 
30 certain intervals. 

A flowchart in FIG. 3 is used to demonstrate how the need for 
guidance is examined and carried out while one is browsing a large docu- 
ment on the display screen of a terminal by scrolling the display. 

Coordinates of the current partial view of the full-size image are 
35 read at the fixed reading point (xi, yi) of the display. In this example, the fixed 
reading point is the middle point of the display. 
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The coordinates of the current view at the middle point of the dis- 
play are then saved at certain time instances e.g. every second, at stage 30. 
With the aid of the set of coordinates, a set of motion vectors is obtained. 
The motion distribution as a time function is calculated at stage 31 by using 

5 the motion vector set. 

At stage 32 the need for guidance is examined. 
If the distribution is narrow, it means that movements have been 
found, which do not deflect much from the main movement direction or 
scrolling direction. In other words, if the scrolling direction is in the direction 

10 of the x-axis, the coordinates of the page, which have been read at the mid- 
dle point of the display, seem to follow the general direction of the x-axis (the 
text line), i.e. the scrolling direction is not exactly rectilinear. If such move- 
ment as described above is not detected or the distribution is relatively 
broad, free scrolling is still allowed, at stage 35. Otherwise, the next stage 33 

15 is to examine more accurately whether there is a need to guide the reading 
direction. In this case the distribution is compared with the saved information 
(see FIG. 2 stage 23) about the direction of the text lines. At stage 34 it is 
analyzed whether the result of the comparison indicates, in a given time, 
successive movements which are roughly in the same direction. If the result 

20 does not indicate such movements, free scrolling of the data is still allowed 
at stage 35. Otherwise, the data is filtered in such a way that the direction of 
the text lines are taken into account at stage 36. Regardless of the result of 
the comparison, the steps above are repeated from stage 30. 

FIG. 4 demonstrates how the defining and filtering are carried out 

25 mathematically. The assumption in this example is that the direction of the 
text lines on the display is defined to be horizontal by the character recogni- 
tion method. A vector 42 represents that direction. The additional assumption 
is that coordinates of the partial view 40 of the full-size image are read at the 
middle point 43 of the display. Based on the successive coordinates, a set of 

30 motion vectors are obtained, wherein if the directions of said motion vectors 
are roughly in the direction of the text line, this indicates that the user is read- 
ing the text lines. The curve 41 represents the set of motion vectors. 

The similarity of directions between the vector 42 and curve 41 is 

examined. 

35 One way of examining the similarity is described in the following. 

Coordinates of the partial view of the full-size image are controlled by a cer- 
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tain time window. In other words while the document is scrolled the position 
of the middle point 43 on the full-size image 10 varies as time function. 

The curve 41 is sampled by a calculating unit using a sample rate 
FS (e.g. FS=1 s). Each sample consists of the coordinates (xi, yi) of a point 
5 Pi on the curve at a given time. For example, a current coordinate pair (xi.yi) 
is saved every second within a certain time interval, such as 10 s. These 
samples formulate a temporal matrix T, where the first row of the matrix 
represents the x-coordinates and the second row the y-coordinates, so that 
each column presents one coordinate pair (xi,yi). 
10 The parameter D represents the direction (angle) of the text line 

under examination. The definition value for D is 0 radians when the text line 
is in the vertical direction and tt/2 radians when the text line is in the horizon- 
tal direction. Of course, parameter D can have any angle value between 0 - 
n2n radians. 

15 The current direction (angle) between each consecutive point in 

temporal matrix T is calculated. For example, if at time t1 the coordinates at 
point P1 are (x1, y1) and at time t2 at point P2 (x2, y1), the vector between 
the two points P1 and P2 defines the direction (angle) at time t1, where At is 
(t2-t1,t2>t1). 

20 In general the result of calculation is N-1 directions or vectors 

when the number of samples is N. The calculated N-1 directions formulate a 
vector U. Here within 10 seconds the result is nine vectors, each of which 
has a direction (angle) between 0-n2n; radians. That is the result is the distri- 
bution of directions within 10 seconds. When the shape of distribution of the 

25 directions is narrow, a lot of movements have been found in the same direc- 
tion. Correspondingly, when the shape of distribution is broad, only some 
movements are in the same direction. 

The similarity of curve 41 and vector 42 is examined by comparing 
the above vectors in the following way. 

30 First a comparison is made as to whether the absolute value of 

the difference between the mean of vector U and parameter D is smaller 
than the predetermined threshold parameter V and whether the standard 
deviation of vector U is smaller than the predetermined parameter X using 
the following equations 

35 Abs(Mean(U) - D) < V (1) 
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Std(U) < X (2) 
If the results are smaller in both comparisons, curve 41 and vector 
42 are in the same direction. Otherwise curve 41 and vector 42 are in differ- 
ent directions. 

5 In the first case mentioned above, i.e. when curve 41 and vector 

42 are in the same direction, the scroll direction is guided to follow accurately 
the reading direction of the text lines by filtering from the scrolling motion 
those components which differ from the direction of the text lines. 

The effect of filtering is seen in FIG. 4 with the help of curve 44 
10 and vector 45. Curve 44 demonstrates movement of the page before filtering 
and vector 45 represents movement of the page after filtering. Thus, filtering 
provides for movement of the portion of view in the full-size image accurately 
in the direction of the text lines. The result is that the user is able to read the 
text lines with ease because unwanted effects are prevented that might 
1 5 make reading uncomfortable. 

One way to carry out filtering is described with reference to FIG. 
5a - 5b and 6a - 6b. 

First, it is assumed that filtering is not in action in FIG. 5a and 5b. 
FIG. 5a illustrates how the view shown on the display looks at time 
20 t1. It is seen that the middle point of the current view 500 is in point P1 at 
time t1. Within time (t1+ At) - 11 the display on the screen has been moved 
or the middle point of the previous display has been moved on the full-size 
image from point P1 to point P2, when the current view 500 is now the view 
501 as shown in FIG. 5b. 
25 FIG. 6a and 6b illustrate a situation similar to what was explained 

above with reference to FIG. 5a-5b, but here the filtering is in action. 

In FIG. 6a the vector a from the initial point P1 to the terminal 
point P2 represents the displacement of the full-size image within a given 
time. The vector b is a unit vector (i.e. | b | =1) in the direction of the x-axis. 
30 The initial point of vector b is at point P1 . 

In this case it is enough to determine only the component of a 
which is parallel with x-axis, because the text lines are in the direction of the 
x-axis. 

FIG. 6b illustrates said determination. First the projection of a on b 
35 is determined by a scalar product: 
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p = a • b = |a||b|cos0 (3) 

where p is the product of the magnitudes of a and b and the co- 
sine angle 9 between them. In this example 6 is tc/4. Then the vector a x 
represents the component vector of a in the x direction, as denoted by 

5 a x = pb (4) 

The initial point of the vector a x is at point P1 and the terminal 
point at point PS. The vector a x represents the displacement of the full-size 
image in the x-direction within the time At. 

The calculation described above is repeated correspondingly for 

10 displacements at intervals as long as the filtering is activated. The time pe- 
riod is a sample period 1/FS. 

The above-described method can be used in a hand-held terminal 
having a motion controlled scrolling operation by adapting the terminal to: 
read coordinates of the scrolling display at a fixed reading point of the screen 

15 at predetermined time intervals, wherein a set of motion vectors is obtained; 
determine the scrolling direction based on the set of motion vectors; examine 
whether the scrolling direction is roughly the same as the direction of a text 
line; and if so, reinforce the displayed scrolling in the direction of the text line 
as long as the difference between the current scrolling direction and the text 

20 line direction is less than a predetermined threshold value. 

The implementation and embodiment of the present invention has 
been explained above with some examples. However, it is to be understood 
that the invention is not restricted by the details of the above embodiments 
and that numerous changes and modifications can be made by those skilled 

25 in the art without departing from the characteristic features of the invention. 
The described embodiment is to be considered illustrative but not restrictive. 
Therefore, the invention should be limited only by the attached claims. Thus 
alternative implementations defined by the claims, as well as equivalent 
implementations, are included in the scope of the invention. For example, 

30 guidance can be implemented in any scrolling direction, also diagonally. 
Parameters such as the sample rate and the number of samples, as well as 
the time intervals, depend on the application used. In some cases it is also 
possible that a user adjusts some of the above parameters, wherein the 
guidance method reacts differently to a slow user and a quick user. 



